At the current state-of-the-art several methods are known with which images or image sequences like videos can be edited with the help of a computer, whereby the state of editing is always indicated. For simplest image editing methods the user must choose, which filter or editing module he wants to apply to the image. There are, for example, diffusing screen modules, sharpness modules and color correction modules, just to name a few. When choosing a filter, then this filter will be used for all image data or image data in a freely definable segment respectively. The resulting image will be computed and displayed. Thus, all image data will be re-computed for the resulting image.
In modern methods of image editing various editing modules can be joined into editing sequences or batch processes, such as an execution sequence, and will then be displayed as the preferred graphics. Utilizing this type of process, the user himself may select the editing module and assemble it like a tree structure. The editing modules then correspond to nodes; the editing modules or nodes respectively can then be linked with one another so that, for example, data from several preceding editing modules flows to succeeding editing modules. This is, for example, the case when data from two different images are to be combined into a complete image. In that case it is also possible for the user to design an editing module and bind it as a so-called plug-in into the program and the execution sequence. In the case of two linked editing modules, the necessary editing steps are first carried out in the execution sequence of the preceding editing module, the entire image data, which is at least partially modified by the editing module, is then transmitted onto the preceding editing module in the execution sequence where it is then also edited.
Similar methods for image editing were also established in the editing of image sequences in video editing programs. Individual image sequences are referred to as streams and, just as two images can be superimposed into one image, several streams can likewise be combined into one stream, when, for example, one image sequence is to be subtitled. In this case, each stream can be edited independent of the other. The above mentioned filter functions can also be applied in this case. Once the editing of such sequences or a combination a sequences is completed, the result will be shown on the screen. Even during editing it is possible to separately indicate the individual steps or the individual image sequences.
In this process, all image data is recomputed if there is any change in the editing parameters, such as the addition of a color filter or a diffusing screen and transmitted downwards in accordance with the tree structure or execution sequence respectively, starting from the module where the change of parameters took place or respectively from the beginning of the globally adjustable parameter and transmitted downward according to the hierarchy. All image data will be recomputed and transmitted downward according to the hierarchy.
Such methods, which allow the user a graphic surface for the editing of video sequences, are described in the U.S. Pat. No. 6,768,499. Editing modules are graphically compiled according to an execution sequence in a hierarchic tree structure. The tree structure comprises nodes, on which e.g. various streams or tracks are combined, other nodes constitute the editing modules. The nodes are connected via the tree structure.
Arrangements using a similar method are described in WO 99/52276. Here, too, several data streams can be edited and combined, whereby the editing can be accelerated by editing several data streams simultaneously. In this way real-time editing should be made possible, i.e. that the results can be immediately shown on the screen and through simultaneous shoots the results obtained can be displayed on the screen without delay.
In both methods and also in the previously described processes, all computations are carried out on the processor motherboard. While the methods described are adequate for two dimensional image contents, one quickly reaches the limit when applying these methods to three-dimensional contents, as the data volume to be edited is many times larger. Although, in principle, two views taken from different perspectives of an object are in order to produce a stereoscopic image or stereoscopic image sequence, which can convey a stereoscopic impression, the viewer's mobility before the screen can be greatly constrained; as a rule, several views are taken from different perspectives or produced retrospectively. Typically eight views are used. According to the known state-of-the-art methods, a real-time display of 3D video editing is not possible, i.e. after carrying out one working step there is always a certain lapse until the result can be shown on the screen. The main problem is dealing with the high data volume.
An attempt at reducing the data volume can be found in US2003/0156822 A1. The reference describes a method of editing image sequences consisting of several tracks or streams respectively. If a segment of such a track is deleted during editing, there will not be any actual deletion of data on the hard disk or the storage medium, but certain markings, so-called pointers, are placed, which point to the respective positions within the track.
The track itself will not be interfered with, which simplifies the data management and the editing time. Image sequences, which are edited in US2003/0156822 A1, concern so-called multi-view streams, i.e. image sequences, which have a common time axis. The contents, however, can be different. It may, for example, concern television images taken during a show of which two are taken of a moderator from different perspectives and a third camera may take the audience.
When using such multi-view streams, it is possible to develop the streams in such a way that each stream shows the same scene, but from slightly different perspective. The data streams of the various views can then be handled and edited and shown as stereoscopic image on a suitable screen. In one of the methods described in WO2006/049384, data streams of all views are edited. After an initial correction eliminates the noise, the camera parameters are determined and a model of the scene is computed. This method provides information on the depth, viewed from the camera position, of the placement of the individual object. From this information, following further editing steps, stereoscopically presentable images can be produced. Transmission on image sequences, however, is cumbersome due to the data volume that needs to be edited and unsatisfactory, as too many computations have to be carried out. Although these computations are necessary in order to be able to produce stereoscopically presentable images, it is not possible to use such pictures or image sequences, if they are edited at a later stage.
The known state-of-the-art methods are hardly suitable for belated image editing and in particular for image sequence for stereoscopic display or multi-view streams respectively, in which filters or functions are to be used on the image data, as it is—even with the currently available efficient processors—not possible to produce the result of the editing in real-time, so that that the viewer has to wait some time for the effect of any changes made by him, i.e. a diffusing screen or color correction. The editing of such image contents is, therefore, very time consuming and expensive.